What is the Cost of Determinism?

We analyze the fundamental performance impact of enforcing a fixed thread communication order to achieve deterministic execution. Our analysis consists of three parts, performed on a real system using the SPLASH-2 and PARSEC benchmarks. First, we quantify the effect that various sources of non-determinism have on program execution. We find that thread communication is the prevalent source of non-determinism and that it sometimes affects program output. Second, we divorce the implementation overhead of a system imposing a specific communication order from the impact of enforcing this order. We show that this fundamental cost of determinism is small (slowdown of 4% on average and 33% in the worst case). Finally, we analyze this cost under perturbed execution conditions. We find that demanding determinism when threads face such conditions can cause almost 2x slowdown.